360 {\ deg}场景中基于图像的显着对象检测（ISOD）对于理解和应用全景信息非常重要。但是，由于缺乏大型，复杂，高分辨率且标记良好的数据集，对360 {\ deg} ISOD的研究尚未被广泛探索。为此，我们构建了一个大型360 {\ deg} ISOD数据集，具有对象级像素的依次投影（ERP），其中包含不少于2K分辨率的丰富全景场景，并且是360 {最大的数据集，是最大的数据集{ \ deg} ISOD据我们所知。通过观察数据，我们发现当前的方法在全景方案中面临三个重大挑战：不同的失真度，不连续的边缘效应和可变的对象量表。受到人类观察过程的启发，我们提出了一种基于样本自适应视图变压器（SAVT）模块的视图显着对象检测方法，并带有两个子模块，以减轻这些问题。具体而言，子模块视图变压器（VT）基于不同种类的变换，在不同视图下学习各种特征，并增强模型的变形，边缘效果和对象量表的特征耐受性。此外，亚模块样品自适应融合（SAF）是根据各种样品特征调整不同变换分支的权重，并使转换的增强功能更适当地融合。 20种最先进的ISOD方法的基准结果表明，构造的数据集非常具有挑战性。此外，详尽的实验验证了所提出的方法是实际的，并且表现优于最先进的方法。

我们提出了Tacobot，这是为首届Alexa Prive Taskbot Challenge构建的面向任务的对话系统，该系统可帮助用户完成多步骤烹饪和家庭装修任务。Tacobot的设计采用以用户为中心的原则，并渴望提供协作且易于访问的对话体验。为此，它具有准确的语言理解，灵活的对话管理和引人入胜的响应生成。此外，Tacobot还以强大的搜索引擎和自动化的端到端测试套件为支持。在引导Tacobot的开发中，我们探索了一系列数据增强策略，以训练先进的神经语言处理模型，并通过收集的真实对话不断改善对话经验。在半决赛结束时，Tacobot的平均评分为3.55/5.0。

文献中有许多不同的方法来解释机器学习结果。但是，这些方法的方法有所不同，通常没有提供相同的解释。在本文中，我们考虑了两种最新方法：集成梯度（Sundararajan，Taly和Yan，2017年）和基线Shapley（Sundararajan和Najmi，2020年）。原始作者已经研究了两种方法的公理属性，并提供了一些比较。我们的工作为表格数据提供了一些有关其比较行为的其他见解。我们讨论两者提供相同解释及其不同的常见情况。我们还使用仿真研究来检查具有Relu激活函数的神经网络拟合模型时的差异。

医疗保健自动化的机会可以改善临床医生的吞吐量。一个这样的例子是辅助工具记录诊断代码时，当临床医生写笔记时。我们使用课程学习研究了医学法规预测的自动化，这是机器学习模型的培训策略，可逐渐将学习任务的硬度从易于到困难提高。课程学习的挑战之一是课程的设计 - 即，在逐渐增加难度的任务设计中。我们提出了分层课程学习（HICU），这是一种在输出空间中使用图形结构的算法，以设计用于多标签分类的课程。我们为多标签分类模型创建课程，以预测患者自然语言描述的ICD诊断和程序代码。通过利用ICD代码的层次结构，该层次基于人体的各种器官系统进行诊断代码，我们发现我们的建议课程改善了基于反复，卷积和基于变压器的体系结构的基于神经网络的预测模型的概括。我们的代码可在https://github.com/wren93/hicu-icd上找到。

本文研究了在潜在的结果框架中使用深神经网络（DNN）的平均治疗效果（ATE）的估计和推理。在一些规则性条件下，观察到的响应可以作为与混杂变量和治疗指标作为自变量的平均回归问题的响应。使用这种配方，我们研究了通过使用特定网络架构的DNN回归基于估计平均回归函数的两种尝试估计和推断方法。我们表明ATE的两个DNN估计在底层真正的均值回归模型上的一些假设下与无维一致性率一致。我们的模型假设可容纳观察到的协变量的潜在复杂的依赖结构，包括治疗指标和混淆变量之间的潜在因子和非线性相互作用。我们还基于采样分裂的思想，确保精确推理和不确定量化，建立了我们估计的渐近常态。仿真研究和实际数据应用证明了我们的理论调查结果，支持我们的DNN估计和推理方法。

与准确性和计算成本具有密切关系的图像分辨率在网络培训中发挥了关键作用。在本文中，我们观察到缩小图像保留相对完整的形状语义，但是失去了广泛的纹理信息。通过形状语义的一致性和纹理信息的脆弱的启发，我们提出了一个名为时间性解决方案递减的新颖培训策略。其中，我们在时域中随机将训练图像降低到较小的分辨率。在使用缩小图像和原始图像的替代训练期间，图像中的不稳定纹理信息导致纹理相关模式与正确标签之间的相关性较弱，自然强制执行模型，以更多地依赖于稳健的形状属性。符合人类决策规则。令人惊讶的是，我们的方法大大提高了卷积神经网络的计算效率。在Imagenet分类上，使用33％的计算量（随机将培训图像随机降低到112 $ \倍112美元）仍然可以将resnet-50从76.32％提高到77.71％，并使用63％的计算量（随机减少在50％时期的训练图像到112 x 112）可以改善resnet-50至78.18％。

我们提出了一种叫做SkullEngine的多级粗内CNN框架，可通过协作，集成和可扩展的JSD模型和三个分段和地标检测细化模型进行高分辨率分割和大规模地标检测。我们在临床数据集中评估了由170 CBCT / CT图像组成的临床数据集，用于分割2骨骼（Midface和Mabless）的任务，并在骨骼，牙齿和软组织上检测175个临床普通的地标。

Unsupervised domain adaptation (UDA) for semantic segmentation is a promising task freeing people from heavy annotation work. However, domain discrepancies in low-level image statistics and high-level contexts compromise the segmentation performance over the target domain. A key idea to tackle this problem is to perform both image-level and feature-level adaptation jointly. Unfortunately, there is a lack of such unified approaches for UDA tasks in the existing literature. This paper proposes a novel UDA pipeline for semantic segmentation that unifies image-level and feature-level adaptation. Concretely, for image-level domain shifts, we propose a global photometric alignment module and a global texture alignment module that align images in the source and target domains in terms of image-level properties. For feature-level domain shifts, we perform global manifold alignment by projecting pixel features from both domains onto the feature manifold of the source domain; and we further regularize category centers in the source domain through a category-oriented triplet loss and perform target domain consistency regularization over augmented target domain images. Experimental results demonstrate that our pipeline significantly outperforms previous methods. In the commonly tested GTA5$\rightarrow$Cityscapes task, our proposed method using Deeplab V3+ as the backbone surpasses previous SOTA by 8%, achieving 58.2% in mIoU.

Different people speak with diverse personalized speaking styles. Although existing one-shot talking head methods have made significant progress in lip sync, natural facial expressions, and stable head motions, they still cannot generate diverse speaking styles in the final talking head videos. To tackle this problem, we propose a one-shot style-controllable talking face generation framework. In a nutshell, we aim to attain a speaking style from an arbitrary reference speaking video and then drive the one-shot portrait to speak with the reference speaking style and another piece of audio. Specifically, we first develop a style encoder to extract dynamic facial motion patterns of a style reference video and then encode them into a style code. Afterward, we introduce a style-controllable decoder to synthesize stylized facial animations from the speech content and style code. In order to integrate the reference speaking style into generated videos, we design a style-aware adaptive transformer, which enables the encoded style code to adjust the weights of the feed-forward layers accordingly. Thanks to the style-aware adaptation mechanism, the reference speaking style can be better embedded into synthesized videos during decoding. Extensive experiments demonstrate that our method is capable of generating talking head videos with diverse speaking styles from only one portrait image and an audio clip while achieving authentic visual effects. Project Page: https://github.com/FuxiVirtualHuman/styletalk.

Witnessing the impressive achievements of pre-training techniques on large-scale data in the field of computer vision and natural language processing, we wonder whether this idea could be adapted in a grab-and-go spirit, and mitigate the sample inefficiency problem for visuomotor driving. Given the highly dynamic and variant nature of the input, the visuomotor driving task inherently lacks view and translation invariance, and the visual input contains massive irrelevant information for decision making, resulting in predominant pre-training approaches from general vision less suitable for the autonomous driving task. To this end, we propose PPGeo (Policy Pre-training via Geometric modeling), an intuitive and straightforward fully self-supervised framework curated for the policy pretraining in visuomotor driving. We aim at learning policy representations as a powerful abstraction by modeling 3D geometric scenes on large-scale unlabeled and uncalibrated YouTube driving videos. The proposed PPGeo is performed in two stages to support effective self-supervised training. In the first stage, the geometric modeling framework generates pose and depth predictions simultaneously, with two consecutive frames as input. In the second stage, the visual encoder learns driving policy representation by predicting the future ego-motion and optimizing with the photometric error based on current visual observation only. As such, the pre-trained visual encoder is equipped with rich driving policy related representations and thereby competent for multiple visuomotor driving tasks. Extensive experiments covering a wide span of challenging scenarios have demonstrated the superiority of our proposed approach, where improvements range from 2% to even over 100% with very limited data. Code and models will be available at https://github.com/OpenDriveLab/PPGeo.